Robust regression with extreme support vectors

Extreme Support Vector Machine (ESVM) is a nonlinear robust SVM algorithm based on regularized least squares optimization for binary-class classification. In this paper, a novel algorithm for regression tasks, Extreme Support Vector Regression (ESVR), is proposed based on ESVM. Moreover, kernel ESVR is suggested as well. Experiments show that, ESVR has a better generalization than some other traditional single hidden layer feedforward neural networks, such as Extreme Learning Machine (ELM), Support Vector Regression (SVR) and Least Squares-Support Vector Regression (LS-SVR). Furthermore, ESVR has much faster learning speed than SVR and LS-SVR. Stabilities and robustnesses of these algorithms are also studied in the paper, which shows that the ESVR is more robust and stable. Regression is an important topic for machine learning. Classification is the special case of regression, in which the outputs are in the set of {0, 1}. Many regression approaches have been proposed, such as Support Vector Regression (SVR) [18] and least squares regression. However, these methods have some drawbacks, e.g., slow learning speed, poor generalization and low robustness [11]. Extreme Learning Machine (ELM) is a successful single hidden layer feedforward neural network for both classification and regression [11]. It has a good generalization with an extremely fast learning speed. Some desirable advantages can be found in ELM, such as extremely fast learning speed and good computational sca-lability. The essence of ELM is that the hidden layer parameters need not be tuned iteratively and the hidden layer's output connection weights can be simply calculated by least squares optimization [10]. ELM has attracted a great number of researchers and engineers recently for their theoretical and application works [13,12,22]. However, the traditional ELM may encounter ill-posed problems and it is difficult to choose appropriate hidden parameters to avoid such problems [14]. Extreme Support Vector Machine (ESVM) [15] is a kind of single hidden layer feed forward network developed from ELM and Support Vector Machine (SVM). It has not only the same advantages as ELM, such as extremely fast learning speed and that hidden layer parameters can be randomly generated, but also a better generalization than traditional ELM on classification tasks due to its output bias term and regularization scheme. It is a special form of regularization networks [5] derived from SVM. ESVM can be also viewed as an approximation method of SVM. Such approximation leads to fast learning speed. Due to these properties, a lot of researches have been conducted …